Skip to content

[Kernel] Honor leading strides in SM90 tensorwise CUTLASS scaled_mm - #55537

Open
jackLei0901 wants to merge 1 commit into
vllm-project:mainfrom
jackLei0901:fix/cutlass-c3x-leading-strides
Open

[Kernel] Honor leading strides in SM90 tensorwise CUTLASS scaled_mm#55537
jackLei0901 wants to merge 1 commit into
vllm-project:mainfrom
jackLei0901:fix/cutlass-c3x-leading-strides

Conversation

@jackLei0901

@jackLei0901 jackLei0901 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Summary

Fix SM90 tensorwise CUTLASS scaled GEMM addressing for aligned sliced or padded
tensor views by forwarding their actual leading strides instead of
reconstructing tightly packed strides from shape alone.

The operator entry point accepts row-major A/output and column-major B with
aligned padded leading dimensions. The common C3x caller discarded that
metadata, and the SM90 FP8 caller independently reconstructed packed strides.
Both cases could silently read or write incorrect addresses. This forwards the
runtime leading dimensions without adding tensor copies; packed inputs retain
the same stride values as before.

The SM90 FP8 caller needs separate handling when swap_ab is enabled: the
row-major output is presented to the epilogue as a column-major transpose, so
its leading dimension belongs to the second stride component.

The SM100/SM120 and blockwise FP8 callers construct their own packed strides
and have the same defect. They are not fixed here because I have no SM100/SM120
hardware to validate on; they are tracked as remaining work on #55534.

The added A leading-stride check does not reject an existing supported packed
layout: B already requires b.stride(1) % 16 == 0, which enforces aligned K
for packed inputs, and packed A has a.stride(0) == K. It only rejects padded
A views whose leading dimension violates CUTLASS's AlignmentAB = 16
requirement. When K itself is unaligned, the Python wrapper already selects
the Triton fallback.

Partially addresses #55534; remaining architecture-specific callers are
tracked on that issue.

Regression coverage

The former combined subset test is expanded across INT8 and FP8, normal and
swap_ab dispatches (M=512 and M=32), and independent padded-A, padded-B,
and padded-output cases. A negative case verifies that a misaligned A leading
dimension is rejected at the operator boundary.

Validation

Environment: NVIDIA H800 PCIe (SM90), CUDA 13.0, PyTorch 2.13.0+cu130.

# Five-way baseline/fix matrix on base a69e75b9
baseline: 8.10% to 8.16% mismatches for each independently padded tensor
patched:  0 mismatches in all five cases; max abs error ~= 3.5e-7

# Final focused regression on base f2e2936f
# (INT8/FP8, normal/swap, A/B/output + alignment)
PYTHONPATH=$PWD /root/autodl-tmp/vllm-fp8-venv/bin/python -m pytest \
  tests/kernels/quantization/test_cutlass_scaled_mm.py -q --disable-warnings \
  -k 'strided_subsets or rejects_misaligned_a_leading_stride'
13 passed, 720 deselected in 3.92s

# SM90 FP8 pre-fix/post-fix probe
before: all 6 combinations incorrect (M=32/512 x padded A/B/output)
after:  all 6 combinations bitwise equal, 0 mismatches

# Full CUTLASS scaled-mm test file on the final revision
PYTHONPATH=$PWD /root/autodl-tmp/vllm-fp8-venv/bin/python -m pytest \
  tests/kernels/quantization/test_cutlass_scaled_mm.py -q --disable-warnings
679 passed, 54 skipped in 1119.16s

# Targeted formatting/lint hooks
uvx pre-commit run clang-format --files \
  csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/cutlass_gemm_caller.cuh
uvx pre-commit run ruff-check --files \
  tests/kernels/quantization/test_cutlass_scaled_mm.py
uvx pre-commit run ruff-format --files \
  tests/kernels/quantization/test_cutlass_scaled_mm.py
All passed

Model evaluations are not applicable: this is a low-level kernel addressing
fix, and the focused numerical comparison covers the affected output contract.

Duplicate-work check

I searched open PRs by issue number and by CUTLASS scaled_mm leading stride
and cutlass_gemm_caller stride. No open PR addresses this bug. The keyword
results #33651 and #41834 concern different kernel/backend changes.

AI assistance disclosure

AI assistance was used to help inspect the CUTLASS/PyTorch stride contract,
prepare the patch and tests, and organize the validation evidence. I reviewed
the changed lines and validation results before submission.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Team

Run ID: 37a6d014-6c3c-4686-9f3d-df3142f8bc46

📥 Commits

Reviewing files that changed from the base of the PR and between 1084637 and 20917ae.

📒 Files selected for processing (4)
  • csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/cutlass_gemm_caller.cuh
  • csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_sm90_fp8_dispatch.cuh
  • csrc/libtorch_stable/quantization/w8a8/cutlass/scaled_mm_entry.cu
  • tests/kernels/quantization/test_cutlass_scaled_mm.py

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Summary

Summary by CodeRabbit

  • Bug Fixes

    • Improved quantized matrix multiplication for padded and non-contiguous tensor views by preserving runtime strides for inputs and outputs.
    • Added validation to reject input tensors with unsupported row-stride alignment.
    • Improved handling of swapped matrix layouts and strided output tensors.
  • Tests

    • Added coverage for strided subsets across integer and FP8 inputs, weights, and outputs.
    • Added validation for misaligned input strides and dtype-specific result tolerances.

Walkthrough

The CUTLASS 3.x scaled matrix multiplication callers now preserve runtime leading strides for supported padded views. Entry points validate A-stride alignment. Tests cover int8, FP8, strided views, and invalid alignment.

Changes

CUTLASS stride propagation

Layer / File(s) Summary
Forward runtime leading strides
csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/cutlass_gemm_caller.cuh, csrc/libtorch_stable/quantization/w8a8/cutlass/c3x/scaled_mm_sm90_fp8_dispatch.cuh
The callers forward runtime leading strides for A, B, C, and D. The SM90 FP8 path selects the correct output stride component when swap_ab is enabled.
Validate aligned input layouts
csrc/libtorch_stable/quantization/w8a8/cutlass/scaled_mm_entry.cu
cutlass_scaled_mm and cutlass_scaled_mm_azp require the A leading stride to be 16-byte aligned.
Validate strided subsets
tests/kernels/quantization/test_cutlass_scaled_mm.py
Parameterized tests cover int8 and FP8 inputs, padded A, B, and output views, and rejection of misaligned A strides.

Estimated code review effort: 2 (Simple) | ~15 minutes

Merge Risk: ⚪ Minimal · up to 20917

This change fixes incorrect scaled GEMM addressing for supported padded tensor views while retaining packed-layout behavior and adds targeted regression coverage. No merge-blocking risk remains.

Sequence Diagram(s)

sequenceDiagram
  participant Test
  participant ScaledMMEntry
  participant GEMMCaller
  participant CUTLASS
  Test->>ScaledMMEntry: submit aligned or padded tensors
  ScaledMMEntry->>ScaledMMEntry: validate A, B, and output strides
  ScaledMMEntry->>GEMMCaller: dispatch scaled matrix multiplication
  GEMMCaller->>CUTLASS: pass runtime leading strides
  CUTLASS-->>Test: write strided output
Loading
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files. (2 skipped: 2 … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR satisfies the addressed requirements in issue #55534: it forwards runtime leading strides for the common C3x and SM90 FP8 callers, handles swap_ab output strides, preserves packed layouts witho…
Out of Scope Changes check ✅ Passed All changes support the linked issue. The A-stride validation and regression tests directly enforce and verify CUTLASS alignment requirements; no unrelated code changes are present.
Title check ✅ Passed The title clearly identifies the main change: honoring leading strides in the SM90 tensorwise CUTLASS scaled_mm kernel.
Description check ✅ Passed The description directly explains the stride-addressing fix, scope, regression coverage, validation results, and remaining architecture-specific work.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 5 functions across 2 files. (2 skipped: 2 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Forward aligned runtime leading strides through the common C3x caller and the SM90 FP8 caller, including its swapped output layout. Add INT8/FP8 regression coverage for normal and swap_ab dispatches plus an explicit alignment rejection test.

Assisted-by: OpenAI Codex <codex@openai.com>
Signed-off-by: jackLei0901 <42642542+jackLei0901@users.noreply.github.com>
@jackLei0901
jackLei0901 force-pushed the fix/cutlass-c3x-leading-strides branch from 1084637 to 20917ae Compare September 6, 2026 08:44
@jackLei0901 jackLei0901 changed the title [Kernel] Honor leading strides in CUTLASS 3.x scaled_mm [Kernel] Honor leading strides in SM90 tensorwise CUTLASS scaled_mm Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant